1. Field of Invention
This invention relates to image-processing systems and methods usable to construct a composite image.
2. Description of Related Art
High resolution machine vision systems, and especially microscopic machine vision systems, often provide a high-resolution, well-focused image of a small portion of a three-dimensional object. However, this is accomplished at the expense of the depth of field, also called the depth of focus, of the surrounding overall image. However, in some machine vision system applications, it is useful both to make high-resolution well-focused microscopic examination and/or record of a small portion of an object and to clearly observe or record the surrounding overall image.
Methods are known for combining, or fusing, separately focused images of a three-dimensional scene or object to accomplish this useful goal. For example, U.S. Pat. No. 4,141,032 to Haeusler discloses focusing through an object at a plurality of levels to produce a plurality of images, filtering each image with a high pass filter, and summing the images passed by the filtering step to produce a composite image containing only the sharp details. Similarly, U.S. Pat. No. 4,584,704 to Ferren discloses obtaining visual in-focus “slices” of an entire image field. When processing the slices as video scan line signals, the signals pass through a high pass filter. This provides signals which may be discriminated to remove all but the spikes, which identify in-focus edges of objects. Signal information between two successive spikes (edges) may also be regarded as part of an in-focus object. The high pass filter may be combined with other filters. The filters may be made adaptive.
In general, such methods that are based on such relatively simple high-pass filtering are relatively fast. However, the composite images obtained are sensitive to the selected filter parameters and/or signal thresholds employed to discriminate between in-focus and out-of-focus features. As such, these methods are not robust for an unpredictable variety of source image objects and/or surface characteristics, and may introduce undesirable information loss and/or admit undesirable high spatial frequency artifacts from out-of-focus image portions.
Methods based on various multi-resolution spatial filtering techniques are also known. For example, U.S. Pat. No. 4,661,986 to Adelson lists numerous papers by Burt describing various aspects of the Burt pyramid method, and discloses a variation of that method. Adelson discloses dividing respective spatial-frequency spectrums of M images into M substantially similar assemblages of N separate specified pixel sample sets that define N spatial frequency bands. A single one of the corresponding samples is selected to derive respective single sets of improved focus pixel samples for each of the N bands. Corresponding pixel samples of the respective single sets are then combined to derive the improved-focus two-dimensional image.
U.S. Pat. No. 5,325,449 to Burt discloses an image fusion method general enough to fuse not only differently-focused images, but qualitatively different images provided by qualitatively different types of sensing/imaging systems. Burt notes that image fusion is successful to the extent that the composite image retains all useful information, does not contain artifacts, and looks natural. Burt notes that previous pyramid techniques, including multi-resolution spatial frequency techniques, have produced noticeable artifacts in composite images. Burt discloses an improved pattern selective method based upon using oriented functions, such as a disclosed gradient function, which improves the retention of edge-like source image patterns in the composite image. This method is enhanced by a local saliency analysis to refine the composite image.
In general, the pyramid methods of Adelson and Burt can be relatively fast, and relatively robust. However, these pyramid methods are primarily based on a single type of mathematical operation which is applied uniformly throughout all images. Thus, the methods remain sensitive, for example, to undesirable high-frequency artifacts which may be produced by unfocused edges in otherwise featureless and/or out-of-focus portions of the source image.
Convolution methods and wavelet-based methods are also known to be usable to derive composite images. Another known “image fusion” method relies on formulating a multi-component energy function and determining the image fusion result which minimizes the energy function. However, all of these methods generally apply a single type of mathematical operation uniformly throughout all images. Depending on the implementation details, such methods are either computationally-intensive, relatively slow, and/or they also share the drawbacks of the previously-described methods.
Further, numerous methods are known that both identify edges in single images and segment single images. However, such methods have failed to consider how to select among multiple source images to create a composite image in a way that anticipates and suppresses the artifacts that are likely to exist in out-of-focus portions of an image.